Skip to content

Conversation

@BiteTheDDDDt
Copy link
Contributor

@BiteTheDDDDt BiteTheDDDDt commented Nov 20, 2025

What problem does this PR solve?

This pull request introduces a new CoordinatorContext class to better manage query coordination and runtime states across BE (Backend) nodes, separating responsibilities from QueryContext. The changes refactor how runtime filter merge controllers and query coordination are handled, improve resource cleanup, and enhance debugging capabilities. The most important changes are grouped below.

Introduction of CoordinatorContext

  • Added new CoordinatorContext class (coordinator_context.h, coordinator_context.cpp) to manage query coordination and runtime states beyond the lifecycle of QueryContext. This class holds references to runtime filter merge controllers and resource contexts, and provides debug and timeout management functions. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-789e0272245b8bd410068d5444860ea98a102ff4ce756f48217d7df53ebd4abfR1-R114), [[2]](https://github.com/apache/doris/pull/58189/files#diff-6839f3ffba20c6a2e7c6727e45c07be21838302c33e196007e8dde8d29b0dd75R1-R94))

Refactoring Query and Coordinator Context Usage

  • Refactored FragmentMgr to create and manage CoordinatorContext instances per query, storing them in _coordinator_context_map, and linking them to their respective QueryContext objects. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R731-R734), [[2]](https://github.com/apache/doris/pull/58189/files#diff-9f558888989dd08ae617466466716e106c8480524f9f1d86d7b0b650115058a1R213-R214))
  • Updated runtime filter merge controller handling to be managed by CoordinatorContext instead of QueryContext, ensuring proper lifecycle and avoiding circular references. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L758-R763), [[2]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L1314-R1336), [[3]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L1351-R1377))

Resource Cleanup and Timeout Management

  • Ensured that CoordinatorContext objects are erased from _coordinator_context_map when a query is cancelled, and added logic to detect and log lingering contexts that have timed out. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R911), [[2]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R973-R986))

Debugging and Monitoring Improvements

  • Enhanced pipeline task dumping to include CoordinatorContext debug information for better observability of query coordination states. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R793-R802), [[2]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L807-L813))

Minor Code Cleanups

  • Removed unused includes and variables from query_context.cpp and updated method signatures for clarity and consistency. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL26), [[2]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL35), [[3]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL46-L49), [[4]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL95-L112), [[5]](https://github.com/apache/doris/pull/58189/files#diff-96a1dcbd297bc02611e5cdcfb8ebcdbde45c2603f22fb3454c0da1c7ec56261aL198-R204), [[6]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R67), [[7]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L78), [[8]](https://github.com/apache/doris/pull/58189/files#diff-9f558888989dd08ae617466466716e106c8480524f9f1d86d7b0b650115058a1R56))

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Nov 20, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

3 similar comments
@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 22.22% (18/81) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.61% (18281/34746)
Line Coverage 38.05% (166319/437057)
Region Coverage 33.00% (129245/391636)
Branch Coverage 33.80% (55472/164116)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 33.33% (27/81) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.30% (19544/34109)
Line Coverage 42.82% (187229/437234)
Region Coverage 37.97% (150712/396957)
Branch Coverage 38.35% (63334/165127)

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 24.42% (21/86) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.64% (18293/34753)
Line Coverage 38.07% (166364/437039)
Region Coverage 33.04% (129377/391631)
Branch Coverage 33.82% (55506/164126)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 36.05% (31/86) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.75% (19358/34111)
Line Coverage 42.20% (184510/437180)
Region Coverage 36.82% (146147/396932)
Branch Coverage 37.39% (61744/165123)

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 46.09% (53/115) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.05% (19461/34112)
Line Coverage 42.54% (185962/437185)
Region Coverage 37.25% (147860/396950)
Branch Coverage 37.80% (62417/165125)

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 46.09% (53/115) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.05% (19461/34112)
Line Coverage 42.53% (185945/437201)
Region Coverage 37.25% (147871/396974)
Branch Coverage 37.79% (62405/165129)

1 similar comment
@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 46.09% (53/115) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.05% (19461/34112)
Line Coverage 42.53% (185945/437201)
Region Coverage 37.25% (147871/396974)
Branch Coverage 37.79% (62405/165129)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 46.09% (53/115) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.16% (19497/34112)
Line Coverage 42.58% (186176/437201)
Region Coverage 37.23% (147784/396974)
Branch Coverage 37.84% (62485/165129)

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 28.87% (41/142) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.83% (18402/34832)
Line Coverage 38.32% (168103/438726)
Region Coverage 33.14% (130554/393975)
Branch Coverage 34.04% (56148/164927)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 86.75% (131/151) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.58% (24475/34191)
Line Coverage 58.07% (254872/438868)
Region Coverage 53.36% (213067/399269)
Branch Coverage 54.73% (90807/165920)

@BiteTheDDDDt BiteTheDDDDt changed the title test [Enchancement](query-context) add coordinator-context to extend lifetime about rf handle Nov 26, 2025
@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 86.75% (131/151) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.52% (24451/34190)
Line Coverage 57.95% (254270/438763)
Region Coverage 53.24% (212505/399155)
Branch Coverage 54.55% (90459/165836)

1 similar comment
@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 86.75% (131/151) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.52% (24451/34190)
Line Coverage 57.95% (254270/438763)
Region Coverage 53.24% (212505/399155)
Branch Coverage 54.55% (90459/165836)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 86.75% (131/151) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.52% (24451/34190)
Line Coverage 57.95% (254259/438763)
Region Coverage 53.23% (212459/399155)
Branch Coverage 54.54% (90447/165836)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants